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ABSTRACT/ 

The purpose of th4s report is to comment on practice 
in the use of existing data bases in program evaluation and school 
improvement and to explore directions of increased and improved use. 
The report's impetus is the role that existing data plays in current 
efforts in local school districts and the possibilities for the 
future. To a great extent, however, we will have to rely on 
experiences from other areas of social inquiry outside of education 
and on national and state practices (rather than local) ill education. 
The record of local district practice in maintaining and using data 
archives is limited to date while extended , ♦.multipurpose secondary 
data examination is commonplace elsewhere. Nonetheless, we assume 
that the current state of affairs in information maintenance -and use 
in local districts is more a happenstance of competing priorities for 
economic and human resources and limited technical expertise and 
support than a conscious judgment that current practices exhaust the 
benefits that might accrue. (Author) 
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The purpose of thfs, report is toiccmment on practice in the use 
of existing data base's -in program evaluation and school improvement 
and to explore directioas of 'increased and improved use. The report's 
impetus is the role that existing data nlays in current efforts in 
local school districts and the possibilities for the future'. To a 
great extent,' however, * we will have to rely on .experiences from other 
areas of social inquiry outside of education and on national and state 
practices (rather than local) in education; The record of local 
district practice' in maintaining and using data archives is limited to 
date while extended, multipurpose»secondary data examination is 
commonplace elsewhere. Nonetheless, we assume that the current state 
of affairs. in informattoajnaintenance and use in local districts is 
more a happenstance of competing priorities for economic and human 
resources and limited technical expertise and support than a conscious 
judgment that current practices* exhaust the benefits that might 
accrue. At a time when the pressures for school improvement are 
strong, and information technology resources are expanding and 
becoming. more cost effective, it seems appropriate to consider the 
coming possibilities. 

Societal Context 

Both recent history and current events regarding public interest 
in education highlight why the question of information maintenance and 
use by school districts is of interest. Consider, for example, the 
issue of possible test score decline and its reflection on the quality 
of schooling which received substantial national attention in the mid 
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70* s. Commentators noted and lamented the apparent decline in * 
performance on the Scholastic Aptijtude Test (SAT). Higher education 
spokespersons were concerned about pre-college preparation and the 
increasing cost*, of remedial instruction that had to come from their 
stagnant or shrinking bases of support. The dissatisfaction of 
industry with the entering skills of the work force was publicly 
expressed. In each, case, the failure of schools to prepare students 
academically to participate fully in society was viewed as a primary 
cause bf the perceived decline in competence. 

Recognizing the need for -better data to inform ptiSlic policy, the 
federal government and private foundations supported several inquiries 
to investigate the following questions: 

1. Was the apparent test score decline real or arti factual ? \ . 

2. If the decline were real, how widespread was it? 

3. If the decline were real, -what were its causes? To what 
degree could it be the result of chaqges in the secondary 

^ school curriculum in the early 7b's (away from traditional 
V courses toward greater . diversity and "relevance"), 

structural changes in expectations regarding the role of 
schools (equity, 'diversity and excellence considerations) and 
broader changes in" the society at large (Ranging nature of 
the family, more single parent homes, more homes with all 
•adults maintaining full -time jobs outside the home, increases 
i n t el e v i s i o n v i ewi n g ) ^ 



Several reports (Harnischfeger S Wijey, 1975; The Wirtz Commission, 

•» 

1977; Munday 1976V and the National. Academy of Education, 1977) 

examined the decline 'issue in detail providing a more balanced picture 

of its extent and its possible antecendents. They generally concluded 

that the decline was not solely the fault of schools as other societal 

« 

institutions and norms had shifted as'well^ These conclusions were 
largely based on available data gathered from SAT and ACT records, 
evidence from state assessments and from scattered research studies 
that focussed'on specific aspects relevant to the test score decline 
<issue. 

What were decidedly absent in the test score decline debate were 
contributions from local school districts. Despite a history of 
annual achievement testing (extending back before the mid 60' s 
compensatory education efforts) and a large expansion, of local 
research and evaluation offices and expertise during the late 60 f s and 
early 70's, LEA 1 s were ill-prepared to contribute to the dialogue. 
Host districts were unable to document whether the national trends 
evidenced in the various policy reports applied to their local 
situation. Instead, they simply had to take the blame along 'with the 

rest of the nation's schools. 

*• * ' ' 

The mid 70's test score decline debate is an -earlier instance 
where local schools were unable to respond to concerns about existing 
practice and performance. Now major national commission reports on 
, the state of education (e.g., from The National Commission on 
Excellence in Education, the National Science Board Cotfimissiori on 



Precollege Education in Mathematics, Science and Technology, the ^ 
Twentieth' Qentury Fund and the Education Commission of the States Task 
Force) place- the spotlight on problems in schools and the needs for 
educational reform* The question is whetner CEAjs will begetter able 
to play an active role in documenting the reform efforts and their 
consequences than they did previously. ■ 
J To a great o extent the ability cf LEA personnel to participate 
depends on the extent to which -they are able to use existing data to 
.guide the evaluation of local programs and inform instructional 
improvement efforts . At present iocil school uses of information 
tends to be responsive and targeted rather than reflective and 
mul tifaceted. Schools routinely coll ect substantial amounts of data 
about the backgrounds, education, interest, performance, and attitudes 
of their students, the employment careers of their personnel, the 
demography of their community , 4 and the- characteristics of their 
educational program. Yet, except for compliance reports for special 
programs, and in some cases monitoring of local school performance, 
'districts seldom retain iriformatipn across years, use data u for 
purposes other than which they were originally collected, or attempt 
to integrate diverse sources of information into a comprehensive 
interlocking system. The question then is whether information 
collected (and in some cases maintained) by local school/s can serve a 
broader array of educational purposes and can contribute to improved 
understanding of the continuities and changes in educational systems 
and their consequences for society* 
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Current General Practices 
There is certainly no shortage of models for broad-based 
maintenance and use of available data. Virtually all aspects of 
American society claim to be data oriented in their decision-making* 
In business and industry, substantial resources are devoted to 
financial analysis, economic forecasting, and marketing studies. 
Medical and health care professionals rely heavily on epidemiological 

data to target^areas of necessary research and to guide practice and 

.. . ft ■ , - 

increasingly monitor routinely collected services and cost 
information. 'Opinion polling and analyses of trends from ongoing 
elections and social surveys have become increasingly central to the 
political process (e.g., in election strategy and establishing, voting 
district boundaries)..' Government agencies place increasing reliance 
on information collection, maintenance and analysis for both its 
monitoring and planning functions. - 

Throughout the social and economic sciences, researchers and 
policy analysts make. extensive use of data collected from major 
social, political and economic surveys. U.S. Census data, the General 
.Social Survey, National ^and Congressional Election Surveys, the 
Criminal Victimization Survey, and the Current Population Survey. are 
examples of primary information resources broadly- investigated ; 
by social scientists. Data from social experiments such as the 
Negative Income Tax Experiments and the Health Insurance Study also 
undergo diverse examination and re-examination. The major q 
investigations in virtually all areas of social inquiry generate data 



bases that are deposited in archives such as those at the 
Inter-University Consortium on Political and Social Research 
(University of Michigan), the Roper Center (University of Wisconsin) 
National Opinion Research Center (University of Chicago) and the 
National Bureau of Economic Research. 

■j . 
? T 

The archiving and use qf existing information resources is deeply 
.ingrained in social research and policy analysis. Yet it is still 
uncommon -to find thoughtful and informative exanri nations of the 
rationale and procedure for data archiving and usage that could serve 
to foster more uniform standards and Expectations and further expand 
the array oj social information, available for further inquiry {an 
early exception is, a book by Gottlieb and Borodim, 1973). ■ < 

The lack of guidelines has been remedied somewhat recently by the 

. _ _ . / . . t * . ■ ■ v 

willingness of various government agency archival and policy analysis 

V * .;.'■':>■ • 

specialists to share their experiences and insights with the social 

/ ; , :-y ; 

research community. A primary example of a new resource o'n support of 
'^proved practice is the compendium on reapalysts of program 
evaluations edited '.by Borucli, Wortman, and Cordray (1981) . This bdok 
contains chapters on the Federal Statistical System, The National 
Archives, The "National* Institute* of Justice's Access and Secondary 
Analysis Policies, and the U.S. General Accounting Office's 
renanalysis activities. It also contains a summary of resources for 
locating public and private data, a description of archiving 
procedures, chapters on pertinent analytical issues in secondary 
analysis and examples of issues where reanalysis series an important 



research or policy purpose. The descriptions of the range of" national 
efforts tp maintain and use data -resources is especially pertinent to 
those areas such as education which have only recently begun to 
establish archiving and usage policies. 1 

Conditions in Education: National and State Practice 
Until the mid 70's, there was a. much thinner record of archiving 
and conducting secondary analyses of data from large-scale educational 
surveys and evaluations. Notable exceptions were the Equality of 
^Educational Opportunity Survey (the basis for the Coleman Report), 
Planned Variation Head Start, The IEA Six Subject Surveys, arxd^roject 
Talent Survey. Otherwise, the data bases from most large-scale 
educational" investigations were used only for the initial inquiry or 
perhaps for follow-up research by the original investigators, and in 
many cases no provisions were made.for retention and maintenance of - 
data resources. ^ • j 

For a variety of reasons, thej situation changed so that multiple « 
uses of existing data bases in educational research, evaluation and 
policy efforts is how more prevalent. The impetus for the change may 



1 One irony of the current status of archival and usage policies in 
education is that the National | Institute of Education supported 
basic conceptual work on multiple uses of existing data (e.g., 
Burstein, 1978; Hendrick, Boruch, & Ross, 1978). In fact, the 
Boruch et al book cited in the test was developed under NIE 
sponsorship. 

£ } This account ignore long established efforts. in Post-Secondary 
4 education to maintain information bases for further research and 

V policy analysis: For example, the Cooperation Institutional v - 
Research Program, has been conducting surveys on entering freshmen 
since the mid 60's under the auspices of the American Council 
Education apd various government. ■...■**..• 
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have .been efforts by the major federal agencies supporting educational 

research and eval uation- to make better use of resources developed 

under their auspices. The National Institute of Education has been 

• most active in their area. Specific examples of recent NIE .practices 

regarding data archiving, maintenance, and usage will help to 

characterize the trends: 

In its Compensatory Education Study to assist in the 
reauthorization of ESEA in 1977, NIE commissioned several policy 
studies which* required analyses of existing data on, e.g., ttie 
consequences of shifting to achievement-based versus 
poverty-based criteria for eligibility for service./ At the time, 
the .task of pulling together diverse data bases from various 
\ . states and government agencies was complicated andjcostly. 

Nonetheless, the political payoff from the stiidy signaled other 
agency personnel of* tl^e "advantages of. maintaining information 
resources for future policy studies. * ' \ 

° The Secondary Analysis Project was supported at Northwestern 
University to accumulate data sets from major education 
investigations and undertake secondary analyses to confirm or 
clarify their findings. The contract also supported the 
publication of a collection of papers (Boruch, Wortmer, Cordray 
and Associates, 1980) on. the state of the art in reanalyzing 
program evaluations. (See earlier description) 

° A grants competition was conducted to encourage secondary 
analyses' of data from the National Assessment of Educational 
Progress (NAEP) (e.g. ,, Harnisch and Linn, 1983.). 

° The contractor for the Beginning Teacher Evaluation Study ( BTES 
Far West Laboratory) was commissioned (under separate agreement) 
to create a data bankv and accompanying documentation for 
follow-up investigations by other researchers. The data were 
deposited in the National Archives and also placed in the ICPSR 
$ata archives. Subsequently, BTES data have been used in 
secondary analyses of both the primary issues in the original 
investigation (e.g., Brown and Saks, 1983) and secondary .issues 
for which subsets of the BTES. data were relevant (e.g. Burstein, 
1980; Maddahian, 1981; Miller, 1981: JWebb, Shave! son & Maddahian, 
1983). 

° The competition for the 1982 award of a new contract for the 

National Assessment of Educational Progress (now managed by. NIE) 
placid increased emphasis on the development of NAEP as a data 
archive for analysis of educational policy and practice. It is V 
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generally believed that the poor track record of the Educational 
Commission of the States in developing NAEP as a national 
resource for policy analysis and the central role of just such a 
purpose in the proposal from the Educational Testing Service was 
a major factor . in the award decision. 

The initial development ^of methodology^ for quantitative 
- integration of analytical data from multiple studies (usually 
.termed meta-analysis. or quantitative synthesis) was almost 
- entirely sponsored by NIE (Glass, 1977; Light, 1978). This 

methodology is now widely used in education and has the secondary 
benefit of encouraging better maintenance and documentation of 
primary research data and highlighting problems with existing 
practices in data collection and reporting. \ 

In recent years, the National Center for Educational Statistics 

(NCES) has^lso be^me more active in ensuring access and use of 

education data bases- Data from the NCES-sponsored National 

Longitudinal Study (NLS) of the High School Class of 1972 has been 

frequently used insecondary analyses of education's meditating role 

in the status attainment process, the more recently sponsored High 

School and Beyond (HS&B) Study is already widely investigated by 

researchers interested in educational policy, most notably in the 

debate on the relative effectiveness of public and private schooling. 

Irf addition to the original report by Coleman, Kil gore and Hoffer 

(1981), there have already been special issues of Harvard Educational 



Review and Sociology of Education stimulated by Coleman et al and 
reporting secondary analyses and reanalyses of the HS&B data. NCES 
has further -encouraged usage of both the NLS and HS&B data through the 
in-house deve^^ment of data archives and documentation for public 
release and the circulation of frequent updates and bulletins about 
these archives. They al so" rec^jil^ of 
HS&B data to identify effective high i^ool^practices. Currently, 
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NCE5 is /taking steps to support the creation of data archives from the 
Second- International Mathematics Study (SIMS) and ensuring that these 
data are readily accessible for analyses beyond the original span of 
interests reflected in the study* 

There has been a similar emergence of data archival- and 
multi-purpose usage by state educational agencies. State assessments, 
originally developed for general and local system monitoring, are 
increasingly used in investigating the impact of new programs 
introduced by the spates, applied research on the characteristics of 
effective school programs, and policy analyses of educational trends * 
and their consequences. 3 Various state educational programs are 
beginning to coordinate their data 'collection efforts to avoid 
redundancy where possible and data archives are being established to. 



We need not look beyond the California Assessment Program (CAP) to 
convey the possibilities in use of state assessment data, CAP data 
have been used in evaluating /the.impact of a state-developed early 
childhood education program (e.g., Baker, 1976;. Isaacs, 1977), the 
effects of compensatory; education services offered in the s'tate 
(California State Department of Education annual reports on * 
evaluation of consolidated application programs (Title I, EDY, 
Miller-Uncuh, ECE), in an effective schools study (California State 
Department of Education, 1976) and in policy studies of the effects 
o f curriculum chan ges and television viewing habits on student 
performance (Harnischfeger and Wiley, 1982). _ ' . 
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ensure ready access to the information collected. 4 

Clearly, national and state educational efforts to develop, 
maintain, and encourage use of existing data resources are evolving. 
The gap in both data resources and expertise in their maintenance and 
use is narrowing between education and other areas of social research 
and services. The shift in attitude from questioning why one would 
want to create data archives and encourage secondary analyses to how 
to best do so is evident. Every major report of the state of 



education calls for greater attention to the need for constant 
maintenance of existing information. ^ The major education agencies at 
the federal and state levels are taking seriously their 
responsibilities. 



4 Again the change in circumstances in California is illustrative. 
As part of an evaluation of the California Early Childhood 
Education Program CSE staff (Keesling & Burstein, 1976} requested 
historical information on program participation, characteristics, 
and performance for a three-year period. At the time, it took 
approximately 12 separate data bases to fulfill the request. In 
essence there was no single data base indicating which schools had 
participated in various compensatory education programs over the 
previous three years and documenting school practices and 
performance.^ Also each year the state constructed a new, separate 
file from its consolidated application reports with no cross-check- 
ing with the previous year's school characteristics and no attempt 
to link performance over time at the school level. In the recent' 
years, however, the California Basic Education Data System (CBEDS) 
was established to routinely collect basic information on student 
enrollments and staff members for use by pVogram personnel at the 
state and local level < and maintain ongoing information about these 
aspects of the state's public schools. : Other efforts have been 
instituted to ensure that up-to-date longitudinal information on 
programs is now readily available.- . 

5 For example, the report of the National Science Board Commission on 
Precollege Education in Mathematics, Science and Technology 
recommends that , r 

The Federal Government shoul d. finance and maintain a national 
' mechanism for measuring student achievement and participation 
in a manner that allows national, state, and local evaluation 
and comparison of education progress. (1982, p. vi) 




- 12 - 



Researchers and evaluators are calling for policies to develop, 
maintain and encourage, multipurpose data usage from sponsored research 
and evaluations and pre-planned secondary analyses (Boruch & Cordray, 
1980; Boruch et al , 1980; Cook and Cruder, 1978, Cronbach. et al , 1980; 
Raizen & Rossi, 1982; Reisner, Al kin, Boruch, Linn, and Millman, 
1982). Evidently, there is hardly anyone left to convince at the 
national and state level (except selected self-interest primary data 
analyst's) that multiple uses of often expensive educational data is 
wise and sensible educational policy and practice, especially at a 
<s>. time of highly visible competing demands for scarce educational 
dollars. 

Local Conditions in Education 
Local school districts are a long way from making full use of 
information from their recurring data collection activities and of the 
improved computer technology for storage, manipulation and access to 
data. However, since LEA's are even more sensitive than state and 
federal agencies to economic conditions and thus have reasons to 
become more cost conscious in their information gathering activities, 
it is reasonable to anticipate growth in multiple uses of existing 
data in these educational settings as well. / 

In most cases, current conditions in LEA yinformation usage 
practices are hardly enlightening or exemplar^. While almost every 
/ district collects standardized achievement t^st data and use it for 
various purposes (e.g., compliance with Title I requirements, public 
reporting of school-level achievement, transmittal to teachers for 
instructional purposes, to principals for class formation and 
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monitoring and to parents for documenting their children's academic 
progress (See Choppin (1982) and Dorr-Brenme (1982) for details of the 
CSE study of test use in schools)) these data represent only a small 
portion of the available information collected apd maintained in some 
fashion by school districts. The types of information routinely 
collected (or collectable) in school districts includes the 
following:** 

s 

A. Demographic/Archival 

1. Student demographics— age, sex, ethnicity, home language, 
parental occupations and employers, eligibility for AFDC, 
reducted price lunches, medical histories, home address, 
mobility (how long in panticul ar residence) parental 
education, family size 

2. Teacher and building-level administrator backgrounds — age, 
education, previous employment and educational history, 
special certification and subject-matter expertise 

3. School building characteristics — information about physical 
plant (e.g., age, capacity, particular resources), 

4. Student body and community composition — ethnic composition, 
neighborhood wealth, community involvement in neighborhood 
schools (e.g., PTA membership) 

B. Financial 

IT Payroll expenditures 

6. Materials and supplies 

7. Equipment 

8. Maintenance 

9. Special programs (e.g. entitlement programs, staff 
development, remedial services, counseling and guidance) 

10. Transportation 

11. Safety and Security 



6 This list is a revision of one provided in Burstein (1983). Also, 
Sirotnik, Burstein, and Thomas (1983) describe the actual data 
routinely collected in seven districts. Other studies (e.g., Bank 
and Williams (1980, 1981), Lyon, Doscher, McGranahan & Williams, 
1978; Kennedy, 1982 ) provide evidence of the data collection 
activities of school districts in the achievement domain only. 
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Testing - \ 

T£Z Standardized norm-referenced tests 

13. Criterion referenced testing 

14. Minimum competency and proficiency testing 

15. Group and individual ability and aptitude testing — done 
typically to determine pupil eligibility for special programs 
and placement decisions 

16. Teacher-made tests and curriculum embedded tests 

Program Characteristics apd Participation 

17. Special program participation « availability and staffing of 
special programs at local school sites 

18. Curriculum information— curricular packages and texts used 
in classrooms, topic coverage from continuum (assumed and 
measured) 

19. Course taking patterns — information from student cummulative 
records and from prescribed offerings 

20. Grading practices — teacher reports of student grades * 

Student Performance, Participation, and Behavior 

21. Grades by content area 

22. Participation in extracurricular activities by types 

23. Awards — e.g., scholarships . 

24. Absenteeism and tardiness 

25. Reported disruptive and inappropriate behavior ^ 

Affective, Attudinal, and Observation Information \ 

26. Student responses to surveys about class and school 
environments and other aspects of their educational experience 

27. Teacher measures of classroom and school climate and 
activities 

23. School building administrator measures of school climate and 
activities 

29. Parental surveys of perceptions and support of school 
activities 

30. Parental participation in school activities (e.g., volunteers, 
fundraising attendance at school functions, scheduled 
conferences) 

31. Administrator observations and evaluations of teachers 

32. Teacher observations of other teachers 

33. District personnel's observation and interviews of building 
personnel 

34. Surveys o graduates to determine occupational and educational 
status 

35. Information about student dropouts 
District Evaluation Reports 

36. Routine annualTeports to board and federal and state agencies 

37. Evaluation of specific educational , changes 

38. Instances of local school assistance by type and disposition 
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The step from simply collecting information to the creation of 
functioning information systems is apparently a giant one. The fact 
that these sources of information are available in school districts 
does not necessarily mean that they are conveniently, accessible or are 

currently monitored for /ends and patterns. There* are a few 

\ 

exemplary efforts by districts to establish comprehensive information 
systems for diverse local use. For example, Cincinnati has had a 
comprehensive multipurpose information system for years. 7 But most 
existing efforts are relatively recent responses to instructional 
improvement and evaluation pressures and again focus almost 
exclusively on achievement test dafcr.7^ 
Functions of Existing District Data 

Obviously the possibilities are known to some districts. It is 
unclear, however, how well actual practices in maintenance and use of 
information systems are documented. Nor the measure necessary to 
disseminate both the benefits of such efforts and the procedures for 
implementing them in other districts well understood. 
It seejris more beneficial at this point to identify the functions 
existing data in local schools* can serve if maintained in accessible 
and documentable form. Furthermore we will comment on the conditions 
pertinent to determining ^hether these functions can actually be 
achieved in local educational settings. The remainder of the report 
will address these two points. 



7 Cincinnati's system is unique in its provision of the information 
and support mechanisms to serve evaluation needs of individual 
schools and attempts to use this information to decentralize educa- 
tional planning. 
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There are at l east five possible functions of existing data in 
local districts, in addition to the direct and immediate uses that 
dictated their collection in the first place* r These functions can be 
termed long-range planning pulse monitoring , student deci si on-maki ng , 
program decision-makin g, and informing educational policy . Bel ow we 
provide brief descriptions of each ''function to enable the reader to 
better understand the distinctions among them. 

Long-Range Planning . y The systematic use of information for 
school district planning is a long-established and well -documented 
purpose for maintaining information. /Decisions about school closings, 
teacher hiring, resource allocation and the like are typically already 
guided by demographica,lly-based planning models. In many respects the 

applications that fall under the rubric of long-range planning 

A 

parallel those in other organizational contexts and as such, have much 
more in common with data use in non-educational organizations than 
they % do with other educational applications. 

Given its linkage to district-wide policy and its centralized 
management, it is not surprising that long-range planning usually 

o . ' ; 

draws primarily on demographic/archival and financial records. . . 
Decisions about school closings for example, depend on trends in local 
enrollments, physical chrapteri sties of .the school sites, 
non-personnel cos1;s (including safety and security factors'; local 

census information about housing'stock ,4nd about changes in ethnic and 

So ■ S / - 
socioeconomic composition, birthrates and projections for future 

growth. If routine community surveys are conducted information 

r 

gathered from this source can also be used. 
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Pulse, Monitoring . In districts already operating information 
systems, the monitoring of datato detect the -general well-being of 
the school system is already common.: These monitoring activities 
• focus on distrijct-wide trends in such areas as student performance, 
graduation rates/^attendance rates, truancy, vandalism, grade 
retentions and attitudes ( students ?1 teachers, parents, community) 
toward aspects of schooling. This user ;of existing data constitutes an 
effort to treat information as "educational indicators" of whether the 
system is moving in desirable directions. In isolated instances, 
pulse monitoring is actually carried out at the school building level 
as well but much more is possible in this area. Cooley (1983) 
provides a thoughtful and informative description of the monitoring 
i function in school districts. £ 

Student Decision Making , According to conventional notions of 
.school guidance counseling, cummulative records of course, taking, 
performance (both tests and grades) and interest 'are used to guide 
educational decisions of students. With the increased possibilities 
,of computerized maintenance of student educational histories, the 
possibilities o.f informed decision making are greatly enhanced. 
Profiles of trends for individual stbdent^ as well as normative T 
patterns for peers and specialized cohorts (e.g. median trends for 
selected set of friends or other with similar educational aspirations) 
can be generated to assist in the decision-making process." 

Program Decision Making . A variety of decisions about 
programmatic changes c'an be informed by existing information bases. 
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Planning for shifts in student interests and capabilities can be 
enhanced by examining data from students at earlier time points. 
Course offerings, year-to-year class assignments, and preparation (of- 
various post-secondary careers can be improved through analysis of 
course-taking patterns, performance, and subsequent "attainment and 
educational decisions from prior cohorts. Similarly, existing data 
can be used to study the consequences of changes in specific programs 

0 

in .individual school sites. 

A clear example of how routinely collected and maintained 
information could be used for program decision making is in the effort 
to monitor the suitability of curriculum offerings in secondary * 
schools. For instance, to determine whether the particular array of 
mathematics courses reflects district goals and student needs, 
information about the distribution of mathematics skills ilpon entry to 
secondary schools, the interests and anticipated educational plans of 
students, the skills believed essential for students to acquire their 
interest (as' defined by district continual, competency and proficiency 
objectives, university enrollment expectations) the alignment of * 

* m ;'/ ' 

special skills to courses, the alignment of various course offerings 

7 

with each other, the availability of personnel and materials to offer_ 

- 7 
all desired courses are likely to be useful. Also it will be 

r 

important to keep track of historical patterns of successful 
completion of courses (e.g., percentage with C or/better, number of- 
students failing specific qourses, specific absenteeism from 
mathematics course rel ative to other classes) to decide whether 
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apparently rational course alignments and,; expectations about\^ 
essential skills are realistic and if not, what can be done tcPiroprove 
matters. Unfortunately this kind of curriculum monitoring has 
apparently not occurred. routinely . Otherwise, the decline in course 
taking in mathematics in secondary schools at a time when the demand 
for technical literacy has increased would have not been allowed to 
happen. Given the renewed emphasis ^on a common academic core at the 
secondary level and the limited demands for data beyond that routinely 
collectable, this area of information use for program decision making 
will surely increase in coming years. 

Informing Education Policy— Once information systems and data 
banks in local districts become. commonplace, it should be possible to 
use them to contribute to the dialogue on state and national policies 
about .the status. of education and the needs and mechanisms for 
improvements. Questions of changes in student performance, community 
attitudes, student interest, curriculum shifts, etc. derived from 
local information systems can be investigated for patterns of local 
reaction to issues of national and state interest. 

Other Possibilities . Two aspects of the designated functions 
^warrant further commen t. Fir ^t, each of the function s i denti f i ed 



typically serves a specific clientele within the educational system. 
Long-range planning is targeted to district-wide management, and falls 
primarily under the auspices of the administration to serve the 
superintendent's efforts to support his' or her educational policies. 
Pulse Monitoring as described here is targeted mor£ at the political 
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community surrounding local education as represented by the school 
board , local media, and comrm -r^" 5 ?-. Tie primary constitu- 

ents for student decision making (students and their parents) and . 
program decision making, (teac^rs and buildings administrators) are 
ob"*ous, t )ugh the onseqi of the decisions serve a wider afray 

of secondary constituents (local business and industry and post-secon- 
dary institutions in the former an4 students and district administra- 
tion in the latter). The clients for the last function virtually 
exhausts the nation's citizenry through the consequences of informed 
educational policy for students and their parents, for educational 
professionals and for the nation's industries, institutions and 
individual taxpayers. ' 

Second, we have purposely excluded two ubiquitous functions f rom . 
our list. Neither research nor evaluation has been explicitly 
mentioned. The availability of information systems in school 
districts can facilitate better ^search on many educational issues. 
But 1c is tl^ secondary benef at accrue from attempts tb serve 

other functions t^at provide both the impetus for educatiopal research 
in schools and often the directions for its improvement. To develop 
local information resources for purely research* purposes runs the risk, 
of creating_ariificial barriers to the informed data use under the • 
mistaken belief that such information surely would not also be useful 
for decisions about educational policy and practice. 

The/avoidance of evaluation as a function is an attempt to move 
away from conventional reliance on such data for summative judgements 
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of program impact, especially at the school and district level. 
Notions of formative evaluation are closer to what/Ts possible with 
existing data, but even this term is limiting. Each of the functions 
identified above relate to some degree to educational decision making 
and description. As such they reflect common aspects of classical 

V.I 

evaluation principles and \models. Yet we view educational processes 
in local educational settings as dynamic, constantly evolving and 
shifting (though often slowly) between various notions of the worth 
and purposes of education and responding like other social 
organizations to both the characteristics of the organizational 
structure and the personalities and intentions of its participants. 
In such circumstances/ it seems best, on the one hand, to characterize 
any activities worthy of the rubric "evaluation" as systemic , to 
convey their interwoven, ongoing and fluid nature or, on the other 
hand, to eschew -evaluation terminology altogether to__avoid static or • 
formal application of evaluation methodology in contexts where it is 
inappropriate. 

Necessary Local Resources Q 

The functions described above fall within the realm of , 
possibility given the intellectual and technological potential for the 
creation, maintenance, and analysis of information already in evidence 
in society* at large. While tjiere are short-term limits to the 
practicality of certain functions of existing data in school 
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districts, 8 none of the ^necessary conditions are intractable with 
sufficient economic resources and interest. 

What local school districts seem to need are the following: 

1. A general committment from the, participants in the 
educational community to the value of existing data for 
informed inquiry and -educational change. * • 

2. Sufficient computer and data literacy on the parts of 
students, teachers, administrators and parents. to benefit 
from the availability of data suitable for informing 
educational decision-making of all kinds. 

■ 0 ' 

3. Computer hardware' and software capable of providing ready and 
. timely access and linkage td information for personnel at the 

school .and district level along with the facility for 
information trapsferral to agenices external to districts for 
* ~broa>3er policy analysis efforts. 



8 For example tire key to a viable function of local i/iformation ■ 
systems for informing broader national and state policies is 
the finding right balance between the desirability/ of comparable 
' reposing categories for multidistrict examinations and the - 
sensibility tailoring* information to local needs. If is obviously 
easier to aggregate across test data when districts use the .same 
tests administers at approximately the same dates. But this is not 
a sensible approach td obtaining data maximally useful for local 
needs arid decision-making. Similarly, while uniform categories for 
vandalism, truancy,, etc. might be desirable for long-term 
monitoring of national trends, the nature of these manifestations- 
of local school problems is' likely to be quite different for'" . 
schools in urban central -city, suburban, and rural-small town. - . 
settings. Some further work is desirable to determine how-to 
develop routine reporting mechanisms that do not place too great a 
demand for across-di strict conformity yet arS viable cross-district 
policy analyses. * / * 
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4. Sufficient computer and data management and analysis 
expertise in school districts (or in networks /of school 

districts) to create, -maintains and analyze data archives and 

^software (including data analysis programs) capable of 
manipulating data to facilitate multiple uses and accomodate 
multiple users. * ' 

5. Sufficient safeguards to ensure protection of personal 
privacy and to assure persons whose information is maintained 
that their privacy is protected. 

- 6. A healthy and informed understanding of, the limits as well as 
the possibilities of information-based decison making. 
Certain aspects of this list warrant further comment. The 
"resources" required cover the range from technological to * 
attitudinal. Certain levels of expertise are obviously necessary as 
are certain amounts of technology. However, three of the resources 
listed are essentially attitudinal or dispositional. Individuals 
affected by the availability and use of information have to believe 
that information can matter (that it can be helpful) and be secure 
from unwarranted invasion of their personal privacy (that information 
will only be used to seisve per^pal and public educational Interests) 
and maintain a healthy scepticism about data-based judgments (.the 
imperfections and margins of error inherent in any data-based 
decision). These attitudinal resources are seldom sufficiently valued 
by those urging improved information use nor are they always actively 
cultivated in educational organizations implementing information 
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systems. One wonders what the marginal benefits and costs would be to 
actively pursue improvements in these areas. 

The area of computing resources, related directly or indirectly 
to points two through/four, could also benefit from further 
elaboration. Currently, there are a variety of efforts to seek to 
incorporate computers as a common multi faceted component of the 
nation's schools (e.g. , .National Science Board Commission on 
Precollege Education. in Mathematics, Science, and Technology, 1983; 
also efforts to provide tax credits for donations of computing " 
equipment t^s^ols). Educators are just beginning^to understand how 
complicated decisions about computing in education can be and the 
costs of insufficient planning for computing needs and practices 
(Becker, 1982a, 1982b; Walker, 1983). 

While it may be some time before sufficient computing resources 
and expertise are available to provide direct instructional access to 

all students, enough computing equipment is probably already available 

V 

to create comprehensive information archives in school districts^ 
Most district already operate main-frame computing machinery which, 
with suitable hardware, and software can be linked to relatively 
inexpensive micro-computers at each of its schools. This linkage will 
place information resources in the hands of building personnel as well! 
and also foster opportunities of improving timely information transferl 
to and from district information banks. 

Within a few years, there will be a sufficient number of 
exemplary on-line information systems operating in school districts to 
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facilitate a major dissemination and diffusion effort through regional 
educational agencies. Problems of technology transfer and adjustment 
to new information environments will undoubtedly remain. But if the 
short-term evidence on the general acceptance of computer literacy as 
an important skill for all students is any indication, it will be more 
a matter of working out the mechanisms for successful implementation 
than justifying the value of the capability. School systems around 
the country are ready to improve their use of information technology. 
All they need now are the means to do ,30. 

Concluding Comments 
The paper is essentially an essay on what local schools can do 
with the information they collect and what conditions need to be 
established to allow them to successfully accomplish, this task. As 
the local schools adapt to the explosion in interest in computers and 
technology, the kind of information system we envision will be a 
natural component of the district's capabilities. Thus better use of 
existing information and better preparedness to anticipate and respond 
to concerns about the well-being of our educational system can be a 
nature by-product of "educating Americans for the 21st Century". 
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